Stable Diffusion

HuggingFace

based on stabilityai/stable-diffusion-2-1#examples on HF
$ pipenv install diffusers transformers accelerate scipy safetensors
in PoC code
- change torch_dtype to torch.float32 (ref)
- change pipe.to parameter to cpu

import torch
from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler

model_id = "stabilityai/stable-diffusion-2-1"

# Use the DPMSolverMultistepScheduler (DPM-Solver++) scheduler here instead
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float32)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to("cpu")

prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt).images[0]
    
image.save("astronaut_rides_horse.png")

Stability-AI GitHub

clone Stability-AI/stablediffusion
download checkpoints (.ckpt and .safetensors files) from HF and put into stablediffusion/checkpoints
`$ pipenv install torch transformers diffusers invisible-watermark omegaconf einops torchvision lightning
$ pipenv install -e .
in stablediffusion/ldm/models/diffusion/ddpm.py comment out import and use of rank_zero_only
in configs/stable-diffusion/v2-inference-v.yaml set use_fp16 to False (mby relevant)
$ python3 scripts/txt2img.py --prompt "a professional photograph of an astronaut riding a horse" --ckpt checkpoints/v2-1_768-ema-pruned.ckpt --config configs/stable-diffusion/v2-inference-v.yaml --H 768 --W 768 --precision full --steps 50 --n_iter 1 --n_samples 1 --device cpu

AUTOMATIC1111 with web UI

based on AUTOMATIC1111/stable-diffusion-webui
# apt install wget git python3 python3-venv
$ bash <(wget -qO- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh) (CAUTION — immediately runs script)
$ ./webui.sh --skip-torch-cuda-test --no-half
then open http://127.0.0.1:7860
extensions
- ControlNet
  - img2img
    - openpose + canny
  - also use script “img2img alternative test”

Stable Diffusion 1.0 XL

Styles (put in styles.csv of webui
(larger collection)

name,prompt,negative_prompt
"Style: Enhance","breathtaking {prompt} . award-winning, professional, highly detailed","ugly, deformed, noisy, blurry, distorted, grainy"
"Style: Anime","anime artwork {prompt} . anime style, key visual, vibrant, studio anime, highly detailed","photo, deformed, black and white, realism, disfigured, low contrast"
"Style: Photographic","cinematic photo {prompt} . 35mm photograph, film, bokeh, professional, 4k, highly detailed","drawing, painting, crayon, sketch, graphite, impressionist, noisy, blurry, soft, deformed, ugly"
"Style: Digital art","concept art {prompt} . digital artwork, illustrative, painterly, matte painting, highly detailed","photo, photorealistic, realism, ugly"
"Style: Comic book","comic {prompt} . graphic illustration, comic art, graphic novel art, vibrant, highly detailed","photograph, deformed, glitch, noisy, realistic, stock photo"
"Style: Fantasy art","ethereal fantasy concept art of {prompt} . magnificent, celestial, ethereal, painterly, epic, majestic, magical, fantasy art, cover art, dreamy","photographic, realistic, realism, 35mm film, dslr, cropped, frame, text, deformed, glitch, noise, noisy, off-center, deformed, cross-eyed, closed eyes, bad anatomy, ugly, disfigured, sloppy, duplicate, mutated, black and white"
"Style: Analog film","analog film photo {prompt} . faded film, desaturated, 35mm photo, grainy, vignette, vintage, Kodachrome, Lomography, stained, highly detailed, found footage","painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured"
"Style: Neonpunk","neonpunk style {prompt} . cyberpunk, vaporwave, neon, vibes, vibrant, stunningly beautiful, crisp, detailed, sleek, ultramodern, magenta highlights, dark purple shadows, high contrast, cinematic, ultra detailed, intricate, professional","painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured"
"Style: Isometric","isometric style {prompt} . vibrant, beautiful, crisp, detailed, ultra detailed, intricate","deformed, mutated, ugly, disfigured, blur, blurry, noise, noisy, realistic, photographic"
"Style: Lowpoly","low-poly style {prompt} . low-poly game art, polygon mesh, jagged, blocky, wireframe edges, centered composition","noisy, sloppy, messy, grainy, highly detailed, ultra textured, photo"
"Style: Origami","origami style {prompt} . paper art, pleated paper, folded, origami art, pleats, cut and fold, centered composition","noisy, sloppy, messy, grainy, highly detailed, ultra textured, photo"
"Style: Line art","line art drawing {prompt} . professional, sleek, modern, minimalist, graphic, line art, vector graphics","anime, photorealistic, 35mm film, deformed, glitch, blurry, noisy, off-center, deformed, cross-eyed, closed eyes, bad anatomy, ugly, disfigured, mutated, realism, realistic, impressionism, expressionism, oil, acrylic"
"Style: Craft clay","play-doh style {prompt} . sculpture, clay art, centered composition, Claymation","sloppy, messy, grainy, highly detailed, ultra textured, photo"
"Style: Cinematic","cinematic film still {prompt} . shallow depth of field, vignette, highly detailed, high budget Hollywood movie, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy","anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured"
"Style: 3d-model","professional 3d model {prompt} . octane render, highly detailed, volumetric, dramatic lighting","ugly, deformed, noisy, low poly, blurry, painting"
"Style: pixel art","pixel-art {prompt} . low-res, blocky, pixel art style, 8-bit graphics","sloppy, messy, blurry, noisy, highly detailed, ultra textured, photo, realistic"
"Style: Texture","texture {prompt} top down close-up","ugly, deformed, noisy, blurry"